Tests and variables selection on regression analysis for massive datasets
نویسندگان
چکیده
Abstract In this paper, a two-stage block hypothesis testing following the idea of Fan, Lin and Cheng (2004) is proposed for massive data regression analysis. Variables selection criteria incorporating with classical stepwise procedure are also developed to select significant explanatory variables. Simulation study confirms that our approach is more accurate in the sense of achieving the nominal significance level for huge data sets. Real data example also verifies that the proposed procedure is accurate compared with the classical method.
منابع مشابه
FARMS: A New Algorithm for Variable Selection
Large datasets including an extensive number of covariates are generated these days in many different situations, for instance, in detailed genetic studies of outbreed human populations or in complex analyses of immune responses to different infections. Aiming at informing clinical interventions or vaccine design, methods for variable selection identifying those variables with the optimal predi...
متن کاملFilter-Type Variable Selection Based on Information Measures for Regression Tasks
This paper presents a supervised variable selection method applied to regression problems. This method selects the variables applying a hierarchical clustering strategy based on information measures. The proposed technique can be applied to single-output regression datasets, and it is extendable to multi-output datasets. For single-output datasets, the method is compared against three other var...
متن کاملFeature selection using genetic algorithm for breast cancer diagnosis: experiment on three different datasets
Objective(s): This study addresses feature selection for breast cancer diagnosis. The present process uses a wrapper approach using GA-based on feature selection and PS-classifier. The results of experiment show that the proposed model is comparable to the other models on Wisconsin breast cancer datasets. Materials and Methods: To evaluate effectiveness of proposed feature selection method, we ...
متن کاملComparison of Ordinal Response Modeling Methods like Decision Trees, Ordinal Forest and L1 Penalized Continuation Ratio Regression in High Dimensional Data
Background: Response variables in most medical and health-related research have an ordinal nature. Conventional modeling methods assume predictor variables to be independent, and consider a large number of samples (n) compared to the number of covariates (p). Therefore, it is not possible to use conventional models for high dimensional genetic data in which p > n. The present study compared th...
متن کاملVariable Selection and Updating In Model-Based Discriminant Analysis for High-Dimensional Data
A model-based discriminant analysis method that includes variable selection is presented. The discriminant analysis model is fitted in a semi-supervised manner using both labeled and unlabeled data. The method is shown to give excellent classification performance on several high-dimensional multiclass datasets with more variables than observations. The variables selected by the proposed method ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Data Knowl. Eng.
دوره 63 شماره
صفحات -
تاریخ انتشار 2007